Examples and Instructions for Context API v2
This guide provides comprehensive examples for using the Context API v2 with detailed explanations of action-specific instructions.
Overview
The Context API v2 introduces structured actions where:
actionsis an object (not an array)- Action names are keys in camelCase format
- Each action's value is an object containing its configuration parameters
This allows for:
- Independent configuration per action
- Type-safe parameter passing
- Clear validation messages
- Better extensibility
Configuration Parameters vs Instructions
Each action has configuration parameters that control its behavior (e.g., maxWordCount, classes, kSimilarMetadata).
Additionally, actions that support an optional instructions field for custom AI guidance:
textSummarization- Guide summarization style and focusimageDescription- Control description format and detail levelnamedEntityRecognitionText- Specify entity types and focus areasnamedEntityRecognitionImage- Guide OCR and entity extractiontextClassification- Provide context for classification decisionsimageClassification- Specify classification focus areastextMetadataGeneration- Guide metadata extraction strategyimageMetadataGeneration- Control metadata identification approach
The instructions field accepts a custom object to provide additional context or guidance to the AI model.
Action Categories
Text Processing Actions
Text Summarization
Generate concise summaries of text documents with optional AI guidance.
Action Name: textSummarization
Configuration Parameters:
maxWordCount(optional): Maximum number of words in the summaryinstructions(optional): Custom instructions object for summarization guidance
Example - Basic:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/report.pdf"
}
],
"actions": {
"textSummarization": {}
}
}
Example - With Word Limit:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/long-report.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 150
}
}
}
Example - Financial Report with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/quarterly-earnings.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 200,
"instructions": {
"focus": "Highlight key financial metrics, revenue, and outlook",
"style": "professional",
"emphasize": "financial_performance|growth_trends"
}
}
}
}
Example - Legal Document with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/contract-terms.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 250,
"instructions": {
"focus": "Extract key obligations, terms, conditions, and deadlines",
"style": "structured",
"highlightRisks": true,
"context": "legal contract analysis"
}
}
}
}
Example - Medical Document with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/patient-report.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 200,
"instructions": {
"focus": "Summarize diagnosis, treatment plan, and follow-up instructions",
"style": "clinical",
"emphasize": "critical_findings|treatment_recommendations",
"context": "medical report summary"
}
}
}
}
Output:
{
"id": "abc123",
"results": [
{
"objectKey": "documents/report.pdf",
"textSummary": {
"isSuccess": true,
"result": "This report analyzes quarterly performance...",
"error": null
}
}
]
}
Text Metadata Generation
Extract structured metadata from text using AI-powered analysis with example-based learning.
Action Name: textMetadataGeneration
Configuration Parameters:
kSimilarMetadata(required): Array of metadata example objects showing desired metadata structureinstructions(optional): Custom instructions object for AI guidance
Metadata Format Guidelines:
- Use
:to separate category from field name (e.g.,"document:type") - Use
|to separate possible values (e.g.,"Invoice|Receipt|Contract") - Provide representative examples of expected values
- Multiple metadata objects help the AI understand variations
Example - Document Classification:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/contract.pdf"
}
],
"actions": {
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"document:type": "Contract|Agreement|NDA|MOU",
"document:category": "Legal|Financial|Employment|Corporate",
"party:count": "2|3|Multiple",
"keywords:tags": "confidentiality|partnership|employment|services"
}
]
}
}
}
Example - Meeting Minutes Metadata:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/meeting-notes.pdf"
}
],
"actions": {
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"meeting:type": "Board Meeting|Team Meeting|Client Call|Review",
"attendees:count": "2-5|6-10|11+",
"has:action_items": "yes|no",
"has:decisions": "yes|no",
"topics:list": "budget|strategy|project_update|personnel",
"urgency:level": "high|medium|low"
}
]
}
}
}
Example - Technical Documentation:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/api-spec.pdf"
}
],
"actions": {
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"document:type": "API Documentation|User Manual|Technical Spec|Guide",
"technology:stack": "REST|GraphQL|gRPC|SOAP",
"programming:language": "Python|JavaScript|Java|C#|Go",
"version:type": "v1|v2|v3|beta|alpha",
"complexity:level": "beginner|intermediate|advanced",
"has:code_examples": "yes|no",
"keywords:tags": "authentication|endpoints|webhooks|integration"
}
]
}
}
}
Example - With Custom Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/legal-agreement.pdf"
}
],
"actions": {
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"document:type": "Contract|Agreement|NDA|MOU",
"document:category": "Legal|Financial|Employment|Corporate",
"keywords:tags": "confidentiality|partnership|employment|services"
}
],
"instructions": {
"focus": "Extract formal legal terminology",
"detailLevel": "high"
}
}
}
}
Output:
{
"id": "def456",
"results": [
{
"objectKey": "documents/contract.pdf",
"textMetadata": {
"isSuccess": true,
"result": {
"document:type": "NDA",
"document:category": "Legal",
"party:count": "2",
"keywords:tags": "confidentiality"
},
"error": null
}
}
]
}
Text Classification
Classify documents into predefined categories.
Action Name: textClassification
Configuration Parameters:
classes(required): Array of at least 2 distinct classification categoriesinstructions(optional): Custom instructions object for AI guidance
Example - Document Type Classification:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/invoice-2024.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"invoice",
"receipt",
"contract",
"report",
"correspondence",
"other"
]
}
}
}
Example - Sentiment Classification:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/customer-feedback.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"positive",
"negative",
"neutral"
]
}
}
}
Example - Industry Classification:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/business-plan.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"technology",
"healthcare",
"finance",
"retail",
"manufacturing",
"services"
]
}
}
}
Example - With Custom Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/legal-contract.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"invoice",
"contract",
"receipt",
"report",
"other"
],
"instructions": {
"context": "legal and financial documents",
"priority": "accuracy over speed"
}
}
}
}
Output:
{
"id": "ghi789",
"results": [
{
"objectKey": "documents/invoice-2024.pdf",
"textClassification": {
"isSuccess": true,
"result": "invoice",
"error": null
}
}
]
}
Text Embeddings
Generate vector embeddings for semantic search and similarity matching.
Action Name: textEmbeddings
Configuration Parameters: None required
⚠️ Breaking Change in v2: The output format for text embeddings has changed from nested array List<List<float>> to flat array List<float>. See Breaking Changes to the Embeddings Format for migration details.
Example:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/article.pdf"
}
],
"actions": {
"textEmbeddings": {}
}
}
Output (v2 - Flat Array):
{
"id": "jkl012",
"results": [
{
"objectKey": "documents/article.pdf",
"textEmbeddings": {
"isSuccess": true,
"result": [-0.021741372, 0.005967313, -0.02682978, 0.01471012],
"error": null
}
}
]
}
Named Entity Recognition (Text)
Extract named entities (people, organizations, locations, etc.) from text documents.
Action Name: namedEntityRecognitionText
Configuration Parameters:
instructions(optional): Custom instructions object for entity extraction guidance
Example - Basic:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/news-article.pdf"
}
],
"actions": {
"namedEntityRecognitionText": {}
}
}
Example - Legal Document with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/contract.pdf"
}
],
"actions": {
"namedEntityRecognitionText": {
"instructions": {
"focus": "Extract party names and signatory individuals",
"entityPriority": "PERSON,ORGANIZATION",
"includeRoles": true,
"context": "legal document with multiple parties"
}
}
}
}
Example - Medical Document with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/medical-report.pdf"
}
],
"actions": {
"namedEntityRecognitionText": {
"instructions": {
"focus": "Extract patient names, doctor names, hospital names, and medication names",
"customEntityTypes": "PERSON,ORGANIZATION,MEDICATION",
"context": "medical report with patient and provider information"
}
}
}
}
Example - Business Email with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/business-correspondence.txt"
}
],
"actions": {
"namedEntityRecognitionText": {
"instructions": {
"focus": "Extract sender, recipient, and mentioned companies",
"prioritizeEmailAddresses": true,
"context": "business communication with contact information"
}
}
}
}
Output:
{
"id": "mno345",
"results": [
{
"objectKey": "documents/news-article.pdf",
"namedEntityText": {
"isSuccess": true,
"result": {
"PERSON": ["John Smith", "Jane Doe"],
"ORGANIZATION": ["Microsoft", "Google"],
"LOCATION": ["New York", "California"]
},
"error": null
}
}
]
}
Image Processing Actions
Image Description
Generate natural language descriptions of images using computer vision and AI.
Action Name: imageDescription
Configuration Parameters:
maxWordCount(optional): Maximum number of words in the descriptioninstructions(optional): Custom instructions object for description guidance
Example - Basic:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product-photo.jpg"
}
],
"actions": {
"imageDescription": {}
}
}
Example - With Word Limit:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/landscape.jpg"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 50
}
}
}
Example - Accessibility Description with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/webpage-screenshot.png"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 100,
"instructions": {
"focus": "Provide alt-text suitable for accessibility",
"detailLevel": "comprehensive",
"includeLayout": true,
"context": "web accessibility for visually impaired users"
}
}
}
}
Example - Product Catalog with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product-item.jpg"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 75,
"instructions": {
"focus": "Emphasize product features, materials, and dimensions",
"includeBrandName": true,
"context": "e-commerce product listing"
}
}
}
}
Example - Medical Imaging with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/xray-scan.jpg"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 150,
"instructions": {
"focus": "Technical medical imaging description for clinical use",
"includeTechnicalTerms": true,
"detailLevel": "high",
"context": "medical imaging documentation"
}
}
}
}
Output:
{
"id": "pqr678",
"results": [
{
"objectKey": "images/product-photo.jpg",
"imageDescription": {
"isSuccess": true,
"result": "A modern smartphone with a sleek black design...",
"error": null
}
}
]
}
Image Metadata Generation
Extract structured metadata from images using AI with example-based learning.
Action Name: imageMetadataGeneration
Configuration Parameters:
kSimilarMetadata(required): Array of metadata example objectsinstructions(optional): Custom instructions object for AI guidance
Example - Product Images:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/clothing-item.jpg"
}
],
"actions": {
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"product:type": "T-Shirt|Dress|Pants|Shoes|Jacket|Hat",
"product:color": "Red|Blue|Green|Black|White|Yellow|Multi-color",
"product:gender": "Male|Female|Unisex",
"product:season": "Summer|Winter|Fall|Spring|All-Season",
"style:type": "Casual|Formal|Sports|Business|Vintage",
"keywords:tags": "fashion|clothing|apparel|style"
}
]
}
}
}
Example - Vehicle Inspection:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/vehicle-inspection.jpg"
}
],
"actions": {
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"vehicle:type": "Sedan|SUV|Truck|Van|Motorcycle|Bus",
"vehicle:color": "White|Black|Silver|Red|Blue|Other",
"vehicle:condition": "Excellent|Good|Fair|Poor|Damaged",
"damage:visible": "yes|no",
"damage:location": "Front|Rear|Side|Multiple|None",
"damage:severity": "Minor|Moderate|Severe|Total",
"environment:location": "Street|Parking Lot|Driveway|Highway|Indoor"
}
]
}
}
}
Example - Real Estate Photos:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/property-listing.jpg"
}
],
"actions": {
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"property:type": "House|Apartment|Condo|Townhouse|Commercial",
"room:type": "Living Room|Bedroom|Kitchen|Bathroom|Office|Exterior",
"style:architecture": "Modern|Traditional|Contemporary|Victorian|Colonial",
"condition:overall": "New|Excellent|Good|Needs Work|Renovation",
"features:notable": "Fireplace|Pool|Garden|Balcony|Hardwood Floors",
"lighting:quality": "Bright|Natural|Dim|Well-lit",
"furnishing:status": "Furnished|Partially Furnished|Unfurnished|Staged"
}
]
}
}
}
Example - With Custom Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product-photo.jpg"
}
],
"actions": {
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"product:type": "T-Shirt|Dress|Pants|Shoes|Jacket|Hat",
"product:color": "Red|Blue|Green|Black|White|Yellow|Multi-color",
"style:type": "Casual|Formal|Sports|Business|Vintage",
"keywords:tags": "fashion|clothing|apparel|style"
}
],
"instructions": {
"focus": "Emphasize brand and quality indicators",
"detailLevel": "high"
}
}
}
}
Output:
{
"id": "stu901",
"results": [
{
"objectKey": "images/clothing-item.jpg",
"imageMetadata": {
"isSuccess": true,
"result": {
"product:type": "T-Shirt",
"product:color": "Blue",
"product:gender": "Unisex",
"product:season": "Summer",
"style:type": "Casual",
"keywords:tags": "fashion"
},
"error": null
}
}
]
}
Image Classification
Classify images into predefined categories.
Action Name: imageClassification
Configuration Parameters:
classes(required): Array of at least 2 distinct classification categoriesinstructions(optional): Custom instructions object for AI guidance
Example - Vehicle Damage Assessment:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/car-photo.jpg"
}
],
"actions": {
"imageClassification": {
"classes": [
"damaged_vehicle",
"undamaged_vehicle",
"not_a_vehicle"
]
}
}
}
Example - Document Type from Image:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/scanned-doc.jpg"
}
],
"actions": {
"imageClassification": {
"classes": [
"passport",
"drivers_license",
"id_card",
"birth_certificate",
"other_document"
]
}
}
}
Example - Scene Classification:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/photo.jpg"
}
],
"actions": {
"imageClassification": {
"classes": [
"indoor",
"outdoor",
"nature",
"urban",
"portrait"
]
}
}
}
Example - With Custom Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/vehicle-damage.jpg"
}
],
"actions": {
"imageClassification": {
"classes": [
"damaged_vehicle",
"undamaged_vehicle",
"not_a_vehicle"
],
"instructions": {
"focus": "Pay attention to minor scratches and dents",
"context": "insurance claim assessment"
}
}
}
}
Output:
{
"id": "vwx234",
"results": [
{
"objectKey": "images/car-photo.jpg",
"imageClassification": {
"isSuccess": true,
"result": "damaged_vehicle",
"error": null
}
}
]
}
Image Embeddings
Generate vector embeddings for image similarity search.
Action Name: imageEmbeddings
Configuration Parameters: None required
⚠️ Breaking Change in v2: The output format for image embeddings has changed from nested array List<List<float>> to flat array List<float>. See Breaking Changes to the Embeddings Format for migration details.
Example:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product.jpg"
}
],
"actions": {
"imageEmbeddings": {}
}
}
Output (v2 - Flat Array):
{
"id": "yz567",
"results": [
{
"objectKey": "images/product.jpg",
"imageEmbeddings": {
"isSuccess": true,
"result": [0.234, -0.567, 0.891, -0.123, 0.456],
"error": null
}
}
]
}
Named Entity Recognition (Image)
Extract text entities from images using Optical Character Recognition (OCR) combined with Named Entity Recognition.
Action Name: namedEntityRecognitionImage
Configuration Parameters:
instructions(optional): Custom instructions object for OCR and entity extraction guidance
Example - Basic:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/business-card.jpg"
}
],
"actions": {
"namedEntityRecognitionImage": {}
}
}
Example - Invoice Document with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/invoice-scan.jpg"
}
],
"actions": {
"namedEntityRecognitionImage": {
"instructions": {
"focus": "Extract company names, invoice numbers, and monetary amounts",
"prioritize": "ORGANIZATION,MONEY,DATE",
"context": "financial document processing"
}
}
}
}
Example - Identification Document with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/drivers-license.jpg"
}
],
"actions": {
"namedEntityRecognitionImage": {
"instructions": {
"focus": "Extract name, address, license number, and expiration date",
"prioritize": "PERSON,ADDRESS,ID_NUMBER,DATE",
"handleMultipleFormats": true,
"context": "identity verification"
}
}
}
}
Example - Receipt with Instructions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/receipt-photo.jpg"
}
],
"actions": {
"namedEntityRecognitionImage": {
"instructions": {
"focus": "Extract vendor name, total amount, date, and item names",
"prioritize": "ORGANIZATION,MONEY,DATE,PRODUCT",
"handleMisalignment": true,
"context": "expense report processing"
}
}
}
}
Output:
{
"id": "abc890",
"results": [
{
"objectKey": "images/business-card.jpg",
"namedEntityImage": {
"isSuccess": true,
"result": {
"PERSON": ["John Doe"],
"ORGANIZATION": ["Acme Corp"],
"EMAIL": ["john@acme.com"],
"PHONE": ["555-1234"]
},
"error": null
}
}
]
}
Complex Multi-Action Examples
Example 1: Complete Document Analysis
Process a document with multiple text actions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/quarterly-report.pdf"
}
],
"actions": {
"textSummarization": {
"maxWordCount": 200
},
"textClassification": {
"classes": [
"financial_report",
"business_plan",
"market_analysis",
"internal_memo"
]
},
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"report:type": "Quarterly|Annual|Monthly|Special",
"report:period": "Q1|Q2|Q3|Q4|FY",
"department:name": "Finance|Sales|Operations|Marketing",
"has:financials": "yes|no",
"keywords:tags": "revenue|expenses|growth|performance"
}
]
},
"namedEntityRecognitionText": {},
"textEmbeddings": {}
}
}
Example 2: Comprehensive Image Processing
Analyze product images with multiple actions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "images/product-photos/item-001.jpg"
},
{
"path": "images/product-photos/item-002.jpg"
}
],
"actions": {
"imageDescription": {
"maxWordCount": 75
},
"imageClassification": {
"classes": [
"electronics",
"clothing",
"furniture",
"food",
"toys",
"other"
]
},
"imageMetadataGeneration": {
"kSimilarMetadata": [
{
"product:category": "Electronics|Clothing|Home|Sports|Beauty",
"product:condition": "New|Used|Refurbished",
"image:quality": "High|Medium|Low",
"background:type": "White|Studio|Lifestyle|Transparent",
"has:watermark": "yes|no"
}
]
},
"imageEmbeddings": {}
}
}
Example 3: Mixed Document Types
Process different document types with tailored actions:
{
"version": "context.api/v2",
"objectKeys": [
{
"path": "documents/contract.pdf"
},
{
"path": "documents/invoice.pdf"
},
{
"path": "documents/report.pdf"
}
],
"actions": {
"textClassification": {
"classes": [
"contract",
"invoice",
"report",
"letter",
"form"
]
},
"textMetadataGeneration": {
"kSimilarMetadata": [
{
"document:category": "Legal|Financial|Administrative|Technical",
"urgency:level": "high|medium|low",
"requires:signature": "yes|no",
"keywords:tags": "important|review|action_required|reference"
}
]
},
"namedEntityRecognitionText": {},
"textEmbeddings": {}
}
}